The Language of Archive between Digital and Material: Mouseion Serapeion and Jaques

The paper indicates multiple theoretical approaches to the studies of archives in order to construct a methodology for studying new media objects and mixed reality phenomena. The theoretical framework is applied to several experimental projects that combine scholarly research with artistic production. VideoSpace is an archive presented as an interactive virtual reality. Mouseion Serapeion is a smart video archive realized as an interactive DVD-ROM and a website. Jaques is an exhibition, which combines three projects: together they address the problem of materialization of the new media object from multiple points of view. 1 The Archival Aesthetics <1> The archive as a specific cultural form seems to be among the key notions in humanities nowadays. Its increased conceptual importance is a result of the fact that for some time now it is no longer possible to believe that meaningful collecting of significant fragments of the past is an unproblematic and self-evident practice. The archive is not merely a container for elements. This insight was illustrated from multiple points of view by many scholars and also by artists. The archive itself intrudes into the collection of elements and thus becomes an additional aspect – or an element – of itself. <2> The key figure in this ongoing process is Michel Foucault, in particular his monograph L'Archéologie du savoir (1969). Reality is understood as an archive, because, as Foucault has argued, the totality of all existing phenomena cannot be reduced to some more or less complex logical schemes, which would potentially explain the heterogeneity of human experience. The ›archaeological‹ approach is also the starting point for his later studies of power relations. Reality is conceptually an archive, because its elements are only ›gathered‹ together, the reason for this is the so-called ›rarity‹ as a property of the concept of discourse: not all the possible results of a conceivable ›logic‹ of a discursive regularity in fact exist, which in turn subverts its logic as such. Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net <3> In the field of new media art and new media communication the theorist Lev Manovich has by now famously argued that a new media object: »consists of one or more interfaces to a database of multimedia material.« The database is in this case a synonym for the archive, a collection of discrete elements. Therefore the new media language deals primarily with archives, which are accessed – by means of interfaces – in meticulously designed ways. In fact, the archives are themselves configured in different ways, since Manovich argues that the new media object – i.e. his main object of research – is, as far as its material existence is concerned, a database. The theory of new media communication systems finds its complement in by now countless projects of digital archives. The famous Media Art Net/Medien Kunst Netz is an example. <4> On the other hand, art – and also the practice of art history – established a fundamental tie with the ›archival aesthetics‹ already in the first half of the 20th century. E.g. Benjamin H. D. Buchloh points to Aby Warburgs Der Bilderatlas Mnemosyne (1924–29), but also to another proliferation of this principle in the sixties, among others in Gerhard Richter’s Atlas (1962–). Buchloh’s ideas are fundamental to the ›institutional critique‹ as the concept for a specific art form or movement. Today the archive is popular among artists as never before and the theoretical ›alliances‹ used spread from Freud to Derrida, Ricoeur, Agamben and many others. <5> To focus the problem, which will be answered by examples of particular artistic archives that were realized by Narvika Bovcon and Aleš Vaupotič with numerous collaborators, the theory of allegory by Walter Benjamin seems a useful starting point. Benjamin explained it in his monograph Ursprung des deutschen Trauerspiels (1928). The text directly speaks about the 17th century counter-reformation ›mourning play‹ (Trauerspiel), however, in this book Benjamin establishes clear parallels between baroque allegory and the expressionist art, too. In his letters to Benjamin Adorno has explicitly established equality between an allegory and an object without the ›aura‹, which becomes the key theme of Benjamin’s essays on technology. The following passage summarizes the problem of the reality which is perceived as an archive: <6> »If the object becomes allegorical under the gaze of melancholy, if melancholy causes life to flow out of it and it remains dead, but eternally secure, then it is exposed to the allegorist, it is unconditionally in his power. That to say it is now quite incapable of emanating any meaning or significance of its own; such significance as it has, it acquires from the allegorist. He places it within it, and stands behind it; not in a psychological but in an ontological sense.« Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net <7> The first sentence stresses that something is terribly wrong with objects: they are »dead«. But at the same time they are »eternally secure«, durable, removed from the natural cycle of life and death. This is the situation, in which »the allegorist« finds himself in. He is faced with an unsolvable problem – if the »gaze of melancholy« could be avoided, then Benjamin’s theory would be pointless, of course – but at the same time he possesses some »power«, some special ability. It could be generalized that any archive (e.g. a museum etc.) confronts the problematic objects, which were torn out of their contexts that gave them life. They lost all their »meaning or significance«. It has to be given to them, and the source of this new meaning – in a world that has lost its immanent significance – is the author-editor of the archive, »the allegorist«. The last sentence of the quote is cryptic. What does it mean that the allegorist gives the things a meaning »in an ontological sense«? It seems that Benjamin’s insight that the source is lost requires another ontology, however the answers that embody it are »practical«. Such as in the examples presented bellow. 2.1 VideoSpace – territory, trajectory, nodes <8> The project VideoSpace (2002/03) is an example of the navigable 3-D computer generated space, in which geometric divisions of the territory reflect conceptual relations among the spatially distributed elements and thus significantly influence the user’s exploration of the space. The triangular conceptual diagram that connects the territory (i.e. the landscape in the virtual space), the trajectory (i.e. the user’s path through the space) and the nodes (the individual artistic projects that are distributed in the space as elements of its database) describes the functioning of the artistically encoded closed virtual space that is to some extent similar to Ilya Kabakov’s concept of the »total installation«. The trajectory is an equivalent of the algorithm that enables ad hoc narrativization of the space, whereas the territory functions as a spatial configuration of the database. The meaning is encoded in the individual elements of the database as well as in their relations, i.e. their vicinity and other relations in the space. We could argue that the spatial distribution of the database elements is also a manifestation of a certain algorithm that triggers narration. The three components acting together build the dramaturgy of experiencing and decoding of the space. Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net 1 N. B., A. V.: VideoSpace Terror, 2003, interactive 3-D space, CD-ROM <9> VideoSpace is a closed archive in which all the elements are connected and they form the whole of a conceptual construct of three basic substances that the world consists of: the human, the language and the matter. Each of the three individual artistic projects (by the same authors N. B. and A. V.) corresponds to one of the substances. The human was the topic of Javornik (2001), where the database of 39 short videos that documented a happening was organized using an image map, i.e. a flat surface that functions as an interface for accessing the videos. The computer screen was divided into nine rectangles with additional four tilted squares on the intersections of the dividing lines. The videos were arranged on three layers and the three videos at the same place on the image map were thematically and visually connected. The image map that guided the organization of the videos was based on the image of the house (with its different parts) where the happening took place. The project that reflected the topic of language was an archive as well: in RIII (2002) Shakespeare’s play Richard the Third was analysed and five constitutive literary motives, themes or features (e.g. the opening soliloquy) were isolated, forming a grid of sources for the subsequent new linear narration (in a video). The third project VSA (2003) dealt with matter: it Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net was an archive of scientific measurements of the cosmic background radiation that were obtained from the Very Small Array telescope of Cambridge University and were displayed in their pure form, as numbers, without the attempt to meaningfully visualize the data. <10> The user brings into the VideoSpace his/her own context (i.e. social, cultural, psychological context, his/her mental archive) that influences his/her understanding of the elements and relations in the VideoSpace archive, but he/she cannot change anything in this virtual space. The conceptual relations between the elements of the archive that are constitutive for any artistic project, and especially for this kind, reveal the problematic nature of Manovich’s proposition (in his book The Language of New Media) that a database as a cultural form is fundamentally not ordered and not hierarchical. The mere contemplation of the elements that are set close to each other in the space – and therefore encountered simultaneously – triggers narrativization at least on the level of their connections, since the user cannot really switch off the understanding of what he/she saw. It turns out that the first stage of narrativization is prompted already by the inclusion of a certain element into a database, which is a consequence of the socio-historical contexts of the archive, and even before that of the contexts from which its elements were taken to be put into the archive. By filling the database we already meaningfully shape the archive. This never happens by itself or outside any context. 2.2 Mouseion Serapeion – archive and interface <11> In VideoSpace the database was relatively small: it consisted of three projects (that were each an archive) segmented into approximately 30 complex elements. Only a relatively small number of elements can be used to determine manageable and meaningful spatial relations that can be grasped by the user, whereas in case of a large database it is not suitable to build orientation in it by means of a virtual space, i.e. to determine the exact and recognizable relationships between all the elements spatially. Large databases that are spread in space limit towards a regular geometric distribution or to chaotic arrangement, however both options are not meaningful enough to build the dramaturgy of its progressive perception. For a large and diverse archive an effective search interface is needed. <12> In the project Mouseion Serapeion (2004) we have attempted to build an interface with a search algorithm that would reshape the configuration of the elements in the archive with each individual search action. Mouseion Serapeion is an archive of videos that were made by students of the Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net Academy of Fine Art and Design at the University of Ljubljana (mostly between 1989 and 1999) and it contains more than a hundred videos. The archive is ›smart‹ because the algorithm remembers the search string for each search and attaches the values attributed to the words that were already searched for to the next searches. In this way the history of browsing is inscribed into the archive. The user’s trajectory through the archive is thus documented, remembered and added to modify the subsequent searches. If we type into the search line of Mouseion Serapeion a certain word now or later on (after a number of search actions), the result of the search will be somewhat different – therefore no two searches are equal from the point of view of the archive configuration. The archive is adapting to the users, it follows their searches, accordingly it enhances the relations between certain elements of the archive that didn't exist a few searches ago and other relations become less important as search in new directions takes place. The elements of the database are not organized once and for all times, but rather they assemble into constellations: by searching the user projects (preliminary) meaning into the archive and by giving it meaning he/she actively reconfigures the archive. At each search the database of the archive is opening up to the user by giving him/her six additional hits besides the one that was searched for, and at the same time it closes down on the user, since only active, diverse searches provide radically new content, whereas if the user searches for similar content, he/she gets similar constellations and to some degree repeated hits. The smart archive models the notion of social and historic identities as they are developed on the level of the atoms of Power-Knowledge in the theory of discourse by Michel Foucault. It also models the techno version of the visage, which confronts the user of the application by means of dialogue; the visage (Emmanuel Levinas) is therefore unknowable in the mystically transcendental sense of the word. <13> The hits and the constellations of related elements are displayed in the ›Territory View‹. There are two views of the interface that accesses the archival database: the ›List View‹ that organizes all the contents of the database chronologically or alphabetically by author, title or genre, and the ›Territory View‹ that displays the seven related elements for each search. The ›Territory View‹ is an endless two-dimensional triangular grid that symbolizes spiritual connections, initiation and contextualization. The computer screen shows only seven nodes in the triangular grid: on these nodes the main hit and six additional hits are placed. The triangular grid represents the spatial mapping of the archive, which is not fixed and the nodes are populated instantly each time when the user enters the archive (i.e. executes a query). Thus also the space of the archive generates itself as a structure in which the connections between any three elements are potentially possible. However, these connections take place only with actual searching. The territory or in other words the space of the archive is configured along the user’s trajectory through the database of the archive. The narrativization of the database that stems out of the relations between the elements of Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net the archive is taking place along the trajectory. 2 N. B., A. V.: Mouseion Serapeion 2.0, 2005, software art, website <14> The project was developed as an interactive DVD-ROM (version 1.0) in 2004 and as an on-line application (version 2.0) in 2005. In January 2006 the internet version was augmented with the Wiki 2.1 version that allows anyone to add new digitized content to the archive, change the existing content or delete it. With the Wiki-version the archive has opened to the users of the web completely. The archive has been conceptualized as an open source software art from the very beginning. The software for the archive can be downloaded from the archive’s web page and used according to a Creative Commons license. The content, i.e. the videos in their entire length, are copyrighted but they are not hidden. <15> The smart archive Mouseion Serapeion was displayed on a computer screen or projected onto an image surface as in the exhibition S.O.L.A.R.I.S., in both cases the flat triangular grid that connects the elements of the archive appeared on a flat screen. In the Mouseion Serapeion 1.1 Installation (2007) the grid was transformed into a three-dimensional constellation of objects in Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net real space at the exhibition 1.3 FVNMA. Each of the seven video hits was displayed on a separate screen in the gallery space, whereas all the videos as well as the archive software were run by a single computer. In this smart exhibition of videos the user browsed through the archive (a selection of digital animations) and played the videos in different constellations, which were in fact video installations. 2.3 Data Dune – Algorithmically generated space <16> The platform Data Dune (2005), conceptualized and designed by Narvika Bovcon, Barak Reiser, Aleš Vaupotič and coded by Igor Lautar, is another example of an interface to an archive that integrates the database with the navigable space. This time the database size is not limited and yet it is presented in a three-dimensional virtual space (whereas Mouseion Serapeion had a flat space, a two-dimensional map). This kind of archival form requires an adequate solution to the problem presented above. <17> Data Dune is an endless virtual space that is generated in real time. It ›accompanies‹ the user as he/she walks through the space by extending itself in the direction of walking. The space is constantly changing also geographically and thus orientation in space and returning to the same spot are virtually impossible. The territory in this space is not a fixed entity, but is being generated according to the user's trajectory. Therefore we cannot use the territory (as we did in the VideoSpace example) to host the elements of the archive at specific geographically determined points and build spatial relations between them that support their conceptual connections. We have used the motif of a desert that to a lost traveller appears endless in all directions and thus disorienting. Also, the terrain changes as the digital dunes travel with the virtual wind. We have based the movement of the dunes in virtual space on the scientific concept of the so-called ›solitary movement of the barchan dunes‹ that emphasises the phenomenon that smaller dunes travel faster than the bigger ones and that dunes preserve their shape as the smaller dune catches up with and travels through a bigger dune. We have recreated the virtual models of the dunes according to the data of scientific measurements of six existing dunes from the barchan dune corridor in Morocco. Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net 3 N. B., Barak Reiser, A. V.: If you look back, it won't be there anymore (on Data Dune platform 1.0), 2005, interactive 3-D space, DVD-ROM <18> Not only is the landscape of the navigable space dependent on the user’s movements in the space, also the apparition of elements from the database of this project follows the user’s expressed interest and activity. Three elements from the database are generated around the user’s position in the virtual space at a radius that is proportional to the distance at which the user experiences the limited visibility near the horizon (we use fog to create the impression of the limited visibility in the atmosphere). At any point in each one third of the circumference of the circle with this radius, one of the three animated objects is generated. The user can now proceed in the direction of any of the three objects that appear on the horizon, or he/she can stroll past all of them into the empty space of the desert. In this second case the same three objects will appear on the horizon again after the user walks the distance of the radius at which the objects are generated. However, if the user visits one or more of these objects, i.e. approaches an object at a very close distance, then after moving away and walking the radius of the apparition, the objects already seen will be replaced by new elements randomly selected from the database. In this way the database Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net detects the user's position in space with help of an algorithm, it counts the objects that were visited and generates new objects following the user's actions. The metaphor of phatamorgana can be recognized as the basis for the construction of the algorithm that displays the elements in this navigable space. The user gets new elements from the archive, if he/she expresses interest in the elements shown, and the space extends itself only as the user moves in it. However, if the user is inactive and doesn’t travel, the dunes still move and the sun light and animated textures that cover the dunes form enough visual changes so that at any time the view in this virtual space functions as a video image. <19> The search algorithm that displays the elements of the database has the function of creating the immersive experience of the virtual space. It builds the techno suspense in the mind of the user who explores the space. It demands activity, otherwise it uncovers nothing new. Finding the elements in the space does not mean systematic searching, as it would be in the case of typing key words in the search line, instead the elements are generated randomly and encountered by strolling in the desert in no particular direction. <20> The Data Dune platform was used for implementation of the artistic project If you look back, it won’t be there anymore (2005-8) for which the set of the elements in the database was conceptualized by Barak Reiser. His set of elements has no time coordinates, instead the narrative that connects the elements unfolds along the formal repetitions, déjà vue, positive and negative forms. Each visited object in the desert space transports the user into a room with a certain theme and encoded meanings that are recognized from our everyday media reality. Switching from the meditative techno suspense in the desert space to the rooms that contain intrusions from our mass-media reality creates flashes of techno surprise that stimulates the user to think about it, to connect the elements of the space, to create possible explanations. <21> The Data Dune platform was used also for an experimental interactive catalogue for the 8 Pixxelpoint – Green Desert (2007), where the objects in the this time green desert space were models of the artistic projects from the festival and by touching them, the texts and other information about the projects appeared overlaid in the space. In the case of the catalogue the database of the elements was determined by the selectors of the festival (N. B. and A. V.). Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net 3.1 Jaques – a new media archive in the mixed reality <22> The notion of the new media archive has been so far explained with three examples of archives that connect navigation in virtual space with a spatially distributed database and with search algorithms that adapt to the user and reconfigure the archive according to the meaning that the user gives it by his/her trajectory (the quasi physical one and the conceptual one). In these cases the archive mostly stayed on the computer as a separate new media object that can in some cases open to the users on the Internet. In the next example we will show the new aspects of the archives that are configured as mixed reality entities. Following the reflection on the reconstruction of a logical unity from a world that has disintegrated into a mass of bordering and permeated archives, a new field of questions opened, i.e. how has the world that surrounds us changed? The theme of the three projects included in the Jaques (2009) exhibition is the entrance of a human with his/her body and a politically charged critical thought into a space that is not homogeneous. The movements around this space mean that movements on the numerous levels of meanings, desires and constraints have to be made. This space is inhabited by people and objects, both of which exist simultaneously in the real world as well as in cyber reality or in some other semiotic reality. The metaphor that was chosen for this experience is Shakespeare’s ›Forest of Arden‹ from the comedy As You Like It. One can enter the Forest of Arden, escape into it, or even permanently live in it, just like the former nobleman Jaques, who has exchanged the aristocratic material possessions for the openness of the abstract forest, where one can »find tongues in trees, books in the running brooks«. 3.2 Double existence of objects – material and digital <23> The project If you look back, it won’t be there anymore is an archive of elements that are shown in the virtual space and are listed in the project book under the ›List of elements‹-section. These objects and their meanings are combined as the user encounters them in the space. However, at the exhibition Jaques the project If you look back, it won’t be there anymore was not just a projection of a self-contained virtual reality, but it functioned also as an archive of different media that were used in its production. Furthermore, it extended its elements from the virtual reality into the real space – forming thus a mixed reality. <24> The digital video Lounge (2008) combines recorded video footage with synthetically produced photorealistic visual material. Digital compositing is the technique that makes simultaneous presence and seamless compositions of altogether different visual material possible. As the Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net metaphor for dwelling in a conceptualized space we have chosen Anton Henning’s installation Frankfurter Salon (2005) in which the three artists spent one afternoon together in the Museum für Moderne Kunst, Frankfurt am Main, when Narvika and Aleš visited Barak in Frankfurt. The Salon is recreated with virtual models in Maya and video footage of the three artists (taken at another meeting in Ljubljana) is composited sitting in Henning’s chairs reading interesting books. The video Lounge starts with the shot of a virtual camera that moves through the Data Dune desert space, following a virtual dragonfly that flies past a solar power plant that powers an electrical neon palm tree. This first part of Lounge is shot exclusively by virtual means. The second part is the mixed media composition of the reconstructed and inhabited Frankfurter Salon, whereas the third part is a documentary video footage of the audience at one of the games of the football championship in Germany in 2006. Although the footage is a recording of the everyday reality, it is recognized as a part of the virtual reality in If you look back, it won’t be there anymore, since the motif of football field is found in the project’s index of elements and recurs at different points in the virtual desert space. <25> The dragonfly that was animated flying through the virtual desert space is present in the gallery space in the form of a sculpture that was designed and constructed as a virtual 3-D model and printed with the 3-D printer used for rapid prototyping. The sculpture Dragonfly (2007) is an example of how information is materialized simply by printing an object. The opposite direction of the flow between the real and virtual objects is used at the other end of constructing the dragonfly: to build a model in Maya we have used an archive of photographs of dragonflies that the biologists use for their study of the different species of dragonflies. In a mixed reality, objects can exist as virtual models and as printed material objects, the transition is easy to achieve, however, the archive of the objects thus doubles into two ontological states with equally important effects on reality: the digital and the material. For example: digitized archives of artistic objects, especially 3D scanned sculptures, can be now accessible also outside museums to a larger scholarly community by printing material copies from the virtual models for a low cost. <26> Another archive has doubled in the present condition of the mixed reality: the archive of real objects, e.g. electrical devices with built-in microcontrollers, with their simultaneous existence in reality and on-line in the frameworks of the ›Internet of Things‹. In one possible future each of our possessions will be found in real space and in cyber space at the same time. This development will result in the fact that these objects and the people who are linked to them will be easily traceable and their actions documented, as everything will be reflected in the changes of the positions and states of the objects. The project IP Light (2008) tests this new condition. It has its own IP Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net address (LAN card) and a built-in microcontroller and thus operates autonomously, just by being plugged into the electricity and connected to the Internet anywhere and with no need for another computer. It can be switched on and off on-line (e.g. with the button on a web page) and also in real space (using a switch on the casing of the light), where the light bulb shines. The light communicates autonomously by storing the information about its status and sending it on request to the application on the web page. The IP Light prototype provides for at least two practical uses: it can be used as a reading light or it can transfer a message as it is turned on and off over the Internet. 4 N. B., A. V.: IP Light, 2008, object, custom hardware and software Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net 3.3 Presence – a smart space prototype <27> In a so-called ›smart space‹ computer vision software is monitoring a social space and detecting changes in human behaviour that differs from the observed regular practices considered typical for this particular social space. We have based our smart space prototype on the theatre as the mediating medium, since it establishes roles, prescribes dialogue and action and builds a stage. We have built an interactive computer installation with two synchronized video streams that are controlled by computer vision software. To manage the action and user responses in this installation we have segmented the gallery space into an archive of zones, further, we have organized different kinds of user behaviour into an archive of behavioural patterns that trigger certain reactions in the installation, finally, the digital avatar on the video projection was animated with a set or an archive of response gestures, and lastly, the computer vision software after recognizing the faces of the visitors in different zones of the space and triggering the corresponding responses from the avatar, recorded all the faces in its database. As we see, the interaction in the smart space prototype is designed as a functional model that combines the above mentioned archives. <28> As encountered increasingly often through the everyday use of smart devices, in the installation Presence (2009) the visitor finds him/herself in a gallery space and in a narrative reality at the same time. He/she takes the place of Lord Buckingham at the dramatic peak of Shakespeare’s play Richard the Third, when king Richard rejects Buckingham’s demand for compensation for his loyal efforts. Thus the visitor replaces the character in the play and the actor on the stage, since he/she is involved in the interactive installation and has to take part in the performance, if he/she wants to get a response from the digital avatar. The visitor has to step into zone A in the space in front of the projections and turn his/her eyes humbly away from the king. Only then the video segments on the projections play in the correct order and reveal a concluding retort of the king. If any of the visitors in the zone A stares at the king disrespectfully, the computer vision software detects the presence of a human face and the video projections become white making the installation disfunctional. We have observed visitors advise each other on the correct behaviour. However, if the visitor steps into zone B, which is too close to (the projection of) the king, the software will detect his/her position and trigger the video fragments that contain alarming warning content. As the visitor turns his/her back to the king and is about to leave the space, maybe being unsuccessful with the interaction before because of the wrong behaviour, on leaving zone A the software will detect a presence and no face and consequently play the dialogue with the king, which will attract the visitor once more to return to interact. Kunstgeschichte. Open Peer Reviewed Journal, www.kunstgeschichte-ejournal.net